Skip to content

feat(scenarios): let an override turn keep-alive off with an empty keepAlive - #53

Open
92Infinitus92 wants to merge 8 commits into
feat/scenarios/override/keep-alive-toggle-supportfrom
feat/scenarios/keep-alive-toggle
Open

92Infinitus92 wants to merge 8 commits into
feat/scenarios/override/keep-alive-toggle-supportfrom
feat/scenarios/keep-alive-toggle

Conversation

@92Infinitus92

@92Infinitus92 92Infinitus92 commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Keep-alive (solana-foundation#906) stays as it is. This PR adds one optional field so a scenario can turn it off for an account, the same way keep_alive: [] does in a template.

OverrideInstance.keepAlive (optional):

  • absent: the template's keep-alive applies, exactly as on main;
  • []: from this override's slot on, nothing rewrites the account it writes, including keep-alive that earlier overrides started, so its fields age as the clock moves on (for example to test a stale price). The override must write a value for this to take effect, like any keep-alive enrollment.

Only [] is supported; any other value behaves as if the field were absent. An account a transaction reads but no scenario names is kept fresh by giving it an override of its own: its template's address: { type: override } import keeps it alive. This was proven with a real JLP deposit on a forked surfnet: with one Doves price override per pool feed the deposit succeeds after the price-age limit, and without the override for one feed it fails with StaleOraclePrice (6003).

Engine change: KeepAliveSource gets an off flag; apply_overrides also records a source when the override sets []; enroll_keep_alive drops every binding whose address is that account and binds nothing for it. Docs: a paragraph in the scenarios README, rpc_endpoints.json, and the generated OverrideInstance.ts (keepAlive?: Array<unknown>).

This replaces the earlier versions of this PR (a boolean flag with an owned address kind, then an override-level import list).

Tests: serde (absent stays absent and is not serialized; [] round-trips); an svm test where a failed write and a non-empty list stop nothing and [] stops keep-alive on the written account only. Core suite 899 passed, types 103 passed, fmt clean, no clippy findings on changed lines.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What it does: adds OverrideInstance.keep_alive (keepAlive in JSON, defaults to true). An override with keepAlive: false writes its account once and does not enroll it in keep-alive.

Assessment: the change is small and correct.

  • The gate in apply_overrides sits inside the scheduled check, so non-scheduled keep-alive rewrites are unaffected.
  • The serde default keeps existing payloads (scenarios, Studio, MCP) working as today.
  • OverrideInstance::new sets the field to true, which matches the serde default.

Issues:

  1. No behavior test. The only test checks serde parsing. Nothing in this repo checks that keepAlive: false actually skips enroll_keep_alive. A small svm-level test would catch a regression in the gate: register a test template with a keep_alive import, apply a scheduled override with keep_alive: false, and assert keep_alive_bindings stays empty. The PR body says the behavior was checked on another branch, but that isn't covered by CI here.
  2. Opt-out does not undo an earlier enrollment. If an earlier override already enrolled the account, a later keepAlive: false override on it has no effect. The PR describes this, but users can easily misread it. Consider logging at debug!/info! when an opted-out override targets an account that is already in keep_alive_bindings, or mentioning it in the field doc.
  3. Unrelated file in the diff. The diff includes .github/workflows/claude-review.yml (141 lines), which has nothing to do with this feature. If the file isn't meant to be part of this PR, it should go in a separate PR.

Comment thread crates/core/src/surfnet/svm.rs Outdated
// if no scenario had touched it.
let mut keep_alive_source = None;
if scheduled
&& override_instance.keep_alive

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gate is correct, but only the serde test covers this PR. Consider an svm-level test: a scheduled override with keep_alive: false on a template that has a keep_alive import should leave keep_alive_bindings empty, and the same override with the default should add a binding. That would catch a regression in this condition.

Comment thread crates/types/src/scenarios.rs Outdated
#[serde(default)]
#[cfg_attr(feature = "ts-bindings", ts(as = "Option<bool>", optional))]
pub fetch_before_use: bool,
/// Whether keep-alive rewrites the account after this override, when its template imports

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider stating here (and in the schemars description) that false only skips enrollment. If an earlier override already enrolled the account, it stays kept alive. Users will likely read keepAlive: false as "stop keeping this account alive". A debug! in apply_overrides for an opted-out override whose account is already bound would also help debugging.

Comment thread crates/types/src/scenarios.rs Outdated
/// Whether keep-alive rewrites the account after this override, when its template imports
/// keep-alive. Off, the override writes once and its fields age as the clock moves on.
#[schemars(
description = "If false, the override writes once and keep-alive does not keep the account fresh afterwards. Defaults to true"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the description says "Defaults to true" in prose. fetch_before_use sets its default with plain #[serde(default)]. If the generated JSON schema should advertise the default, add #[schemars(default = "default_keep_alive")] so MCP clients see it in the schema itself.

@92Infinitus92 92Infinitus92 changed the title feat(scenarios): let an override opt out of keep-alive feat(scenarios): keep-alive for every account of a kind, and per-override stop and restart Nov 9, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR adds a per-override keepAlive flag (default true). Serde, the TS types, rpc_endpoints.json and the README are all updated for it.

Scope and description mismatch

The PR does more than the title and body say:

  • Commit bafeba3 adds a new feature that the description never mentions: the owned keep-alive address type, KeepAliveKind, account_discriminator, YamlOwners and an IDL check in check_keep_alive_imports. This is a separate feature and deserves its own description, or its own PR.

  • The CI workflow is unrelated: .github/workflows/claude-review.yml should not ship in this PR.

  • The description contradicts the code. The body says "The flag only decides whether this override enrolls its account. An account that another override already enrolled stays kept alive." The code does the opposite:

    • keepAlive: false drops every existing binding for that address.
    • It also adds the address to excluded, which blocks future enrollment from other overrides and filters it out of owned kinds.

    Either the description or the code is wrong. The README and tests match the code, so the description is probably out of date.

Main concerns

  • Re-enabling does not restore input/PDA bindings. See the inline comment on keep_alive.rs:66.
  • Per-slot cost of owned kinds. Every slot, get_account_owned_by clones every account of each owner. If one read fails, the whole owner is skipped for that slot.

Tests cover serde defaults, exclusion and re-inclusion through owned kinds, and the "failed write excludes nothing" path. None of them shows that a plain input binding comes back after re-enable, which is the gap described above.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
for source in touched {
if !source.keep_alive {
self.bindings
.retain(|binding| binding.address != source.address);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-enabling does not restore dropped bindings. retain permanently drops every binding whose address is this account, including bindings created by a different importing template (for example, the vault's input: oracle import).

A later keepAlive: true override on the same account only removes it from excluded (line 72). It does not restore those bindings. Keep-alive resumes only if:

  • the original importing override (the vault) plays again, or
  • an owned kind happens to cover the account.

The test at line 389 only passes because oracle is reached through the owned kind, not through its binding.

This contradicts the README ("A later override with keepAlive: true … on that account starts it again"). Two possible fixes:

  • keep the bindings and filter excluded in targets(), as is already done for owned kinds; or
  • make the docs say that re-enabling needs the importing override to replay.

Add a test that uses an input binding only, with no owned kind.

Comment thread crates/core/src/scenarios/README.md Outdated
or its owner with its owned accounts, is marked offline. Keep-alive is kept in memory only: after a restart on a persistent database it stops
never takes effect. An override with `keepAlive: false` stops keep-alive for the account it writes
from that slot on, so the account ages as the clock moves on, for example to test a stale price. A
later override with `keepAlive: true`, the default, on that account starts it again. An account not

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is inaccurate for input/PDA bindings, as explained on keep_alive.rs:66. Re-enabling only clears the exclusion. A binding that was dropped is not recreated unless the importing override plays again.

The PR description also says the opposite: that an account another override already enrolled stays kept alive. Please align the description, the docs and the code.

Comment thread crates/core/src/surfnet/svm.rs Outdated
unix_time_ms: self.updated_at,
};
let targets = self.keep_alive.targets(template_registry(), |owner| {
self.get_account_owned_by(owner).unwrap_or_else(|e| {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per-slot cost and error handling for owned kinds.

This runs every slot for each owner of each owned kind. get_account_owned_by clones every account (full data) of the owner just to check an 8-byte prefix. keep_alive_overrides then calls get_account on each target again. On a busy fork, for example the Pyth receiver program with many price accounts loaded, this adds a full scan of that owner's accounts to every slot.

Also, get_account_owned_by collects into a Result. If any single account read fails, keep-alive silently skips all of that owner's accounts for the slot.

Suggested fix: iterate accounts_by_owner directly and check the discriminator on borrowed data, skipping only the accounts that fail to read.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR adds an owned keep-alive address kind, which rewrites every local account of the given owners that carries an IDL account discriminator. It also adds a per-override keepAlive flag that stops and restarts keep-alive for one account. The per-slot logic now lives in KeepAlive::enroll and KeepAlive::targets, so it can be tested without the bundled registry. That is a good refactor, and the tests cover the stop and restart cases well.

Assessment

The design is sound. The points below are mostly about edge-case semantics and per-slot cost:

  • Unrelated CI file. The PR also adds .github/workflows/claude-review.yml in commit e15c9a0. It has nothing to do with scenarios and would be better as its own PR.
  • Per-slot cost of owned kinds. Every slot, targets calls get_account_owned_by for each owner of each kind. That clones every account the owner has locally, including its full data, only to check an 8-byte prefix. keep_alive_overrides then reads each matching account again through get_account. If two kinds or imports share an owner, the owner is scanned twice. On a fork that has loaded many accounts of one program, this runs on every slot. Some fixes: cache results per owner within one targets call, have the lookup closure return only matching pubkeys, or skip the second get_account for owned targets, since they are known to be local.
  • Exclusion is per address, not per import. keepAlive: false on an account stops every keep-alive template that writes it. That matches the docs. It also means one override can't keep one template, such as freshness, while stopping another. That's fine, but it's worth one sentence in the README.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
/// An import that rewrites a stopped account is bound too, and starts when the account does.
pub fn enroll(&mut self, registry: &TemplateRegistry, touched: Vec<KeepAliveSource>) {
for source in touched {
if !source.keep_alive {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When keep_alive is false, this continues before the source template's own imports are bound, and it excludes source.address itself. For an importing override, such as a vault with keepAlive: false, that means two things:

  • The vault is excluded, but keep-alive never writes it anyway.
  • Its oracle imports are never bound, so a later keepAlive: true on the oracle restarts nothing for this vault.

That clashes with the "off first" behaviour in the docs, where bindings are kept so a restart covers them. It would be more consistent to bind the imports regardless and only add the address to excluded. Otherwise, document that keepAlive: false on an importer also skips its imports.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
for binding in &self.bindings {
if let Some(import) = import_of(&binding.template, binding.import)
&& !self.excluded.contains(&binding.address)
&& seen.insert((import.template.as_str(), binding.address))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deduplication is keyed by the keep-alive template and address only. Two imports can use the same keep-alive template with different values on one account, for example posted_slot: $slot and posted_slot: $slot-5, or different field sets. In that case the first one in BTreeSet order wins and the other is dropped silently. Consider keying by the import, meaning (template, index), or logging when a conflicting import is dropped.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
continue;
};
for owner in owners.iter().filter_map(|owner| owner.parse().ok()) {
for (address, owned) in accounts_owned_by(&owner) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scans each owner once per kind, and the closure clones every owned account, including its full data, every slot. If several kinds or imports name the same owner, the scan repeats. Cache accounts_owned_by(owner) per call, or change the closure to take the discriminator and return only matching pubkeys, so it doesn't clone data it never uses.

Comment thread crates/core/src/surfnet/svm.rs Outdated
};
let missing = match self.get_account(&binding.address) {
for (import, address) in targets {
let missing = match self.get_account(&address) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Owned targets come from get_account_owned_by, so they are already known to be local. This get_account reads each one a second time every slot. Passing the account, or a local flag, out of targets would avoid the duplicate lookup.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The PR adds an owned keep-alive address kind, which rewrites every local account of an owner/IDL account type each slot. It also adds a per-override keepAlive flag that stops keep-alive for an account and later starts it again. The per-slot logic moves into KeepAlive::enroll and KeepAlive::targets, which can be tested without the bundled registry.

The design is clean and well tested:

  • A failed write excludes nothing.
  • Bindings are kept while an account is excluded, so turning keep-alive back on restores imports from other templates.
  • An account bound by two imports of the same keep-alive template is written once per slot.
  • check_keep_alive_imports checks that the account exists in the keep-alive IDL.
  • reset_network clears everything.

Points to consider

  • Per-slot cost of owned kinds. targets calls get_account_owned_by for each owner every slot, which clones every account that owner owns. keep_alive_overrides then re-reads each matching account and builds an IDL-based override for it. If a client has pulled a large program's accounts (for example via getProgramAccounts on the Pyth receiver), this can add up to thousands of clones and rewrites per slot. Possible mitigations: filter on the discriminator before cloning, or cap or log the number of owned targets.
  • One bad account blanks a whole owner. get_account_owned_by collects Results, so one unreadable account makes it return an error. The closure then turns that into Vec::new(), and every account of that owner is skipped that slot, not just the broken one.
  • Exclusion is per account, not per template. keepAlive: false on an oracle stops every keep-alive template that writes that address. That is what the docs describe, but it's worth stating explicitly in the README.
  • Unrelated file. .github/workflows/claude-review.yml is part of this diff (commit e15c9a0) and has nothing to do with the feature. Consider splitting it out.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
continue;
};
for owner in owners.iter().filter_map(|owner| owner.parse().ok()) {
for (address, owned) in accounts_owned_by(&owner) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This runs every slot and asks for every local account of each owner, fully cloned, before filtering on the discriminator. With many accounts under a program like the Pyth receiver, that is a lot of cloning plus one IDL rewrite per match on every slot. Could the lookup return only the pubkeys and the 8-byte prefix, or filter inside get_account_owned_by? A cap or a warning when an owned kind reaches more than N accounts would also help.

Comment thread crates/core/src/surfnet/svm.rs Outdated
unix_time_ms: self.updated_at,
};
let targets = self.keep_alive.targets(template_registry(), |owner| {
self.get_account_owned_by(owner).unwrap_or_else(|e| {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_account_owned_by collects into Result<Vec<_>>, so one unreadable account makes it return an error. This closure then turns the error into Vec::new(), and every account of that owner misses keep-alive for the slot. Consider skipping only the account that fails, for example with a filter_map that logs the error.

pub streamed_accounts: Box<dyn Storage<String, bool>>,
pub recent_blockhashes: VecDeque<(SyntheticBlockhash, i64)>,
pub scheduled_overrides: Box<dyn Storage<u64, Vec<OverrideInstance>>>,
/// The accounts keep-alive rewrites every slot, from the slot a scenario's override first

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc comment is out of date now that the field also holds owned kinds and the excluded set. It should say the field covers bindings, owned kinds and accounts excluded by keepAlive: false.

Comment thread crates/core/src/scenarios/keep_alive.rs Outdated
}
continue;
}
self.excluded.remove(&source.address);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This removes the exclusion before the template lookup, so an override with keepAlive: true on an unknown or import-less template still restarts the account. That is intentional and covered by tests. A short comment here saying the restart does not depend on the template having imports would stop someone from moving it below the registry.get early-continue later.

Comment thread crates/types/src/rpc_endpoints.json Outdated
"label": "Option<String> (An optional label for this override instance)",
"enabled": "bool (Indicates whether this override instance is enabled)",
"fetchBeforeUse": "bool (Indicates whether to fetch the latest on-chain account data before applying overrides)",
"keepAlive": "bool (Optional, defaults to true. If false, keep-alive stops rewriting this account from this slot on, and its fields age as the clock moves on. A later override with true starts it again)",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this, why have we rewritten the entire concept and logic of keep_alive and turned it into boolean ???

@92Infinitus92 92Infinitus92 changed the title feat(scenarios): keep-alive for every account of a kind, and per-override stop and restart feat(scenarios): let an override pass its own keep-alive imports Nov 20, 2026
@92Infinitus92 92Infinitus92 changed the title feat(scenarios): let an override pass its own keep-alive imports feat(scenarios): let an override turn keep-alive off with an empty keepAlive Nov 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants